[Chapter 23] 23.6 Cause the Queue to Be Processed
Author: Bryan Costales & Eric Allman
Date: 1999-01-06T19:34:05Z

#

| Chapter 23 The Queue |

## 23.6 Cause the Queue to Be Processed

The sendmail program offers two different methods for processing the queue. It can be told to process the queue periodically or to process it once and then exit.

### 23.6.1 Periodically with -q

The `-q` command-line switch is used both to cause the queue to be processed and to specify the interval between queue runs.

A typical invocation of the sendmail daemon looks like this:

>     /usr/lib/sendmail -bd -q1h

Here, the sendmail program is placed into listening mode with the `-bd` command-line switch. The `-q1h` command-line switch tells it to process the queue once each hour. Note that either switch puts sendmail into the background as a daemon. The `-bd` switch just allows sendmail to listen for incoming SMTP connections. Consider the following:

>     /usr/lib/sendmail -bd
>     /usr/lib/sendmail -q1h

This runs two daemons simultaneously. The first listens for incoming SMTP connections. The second processes the queue once per hour.

At small sites, where mail messages are rarely queued, the time interval chosen may be small, to ensure that all mail is delivered promptly. An interval of `15m` (15 minutes) may be appropriate.

At many sites an interval of one hour is probably best. It is short enough to ensure that delays in delivery remain tolerable, yet long enough to ensure that queue processing does not overlap.

At large sites with huge amounts of mail and at sites that send a great deal of international mail, the interval has to be carefully tuned by observing how long it takes sendmail to process the queue and what causes that process to take a long time. Points to consider are the following:

- Network delays or delays at the receiving host may cause delivery to that host to time out. Timeouts are set with the `Timeout` (`r`) option (see Section 34.8.70). [5] Each such timeout is logged at LOG_NOTICE with a message like this:

  > [5] Note that prior to V8 sendmail the `r` option set one timeout for all SMTP timeouts.

  >     timeout waiting for input from host during what

  Here, host is the name of the other host, and `what` specifies which timeout triggered the message (such as "client HELO for `to_helo`). In general, timeouts should be large to ensure that mail to busy sites and to large mailing lists does not time out improperly. In observing the queue processing, you may find that all messages but one process swiftly. That one, you may find, takes over an hour because of a long SMTP timeout. A possible solution to this problem is to make all timeouts short so that most queue runs are processed quickly. Then, for example, the following command could be run a few times each night to specifically flush those long jobs:

  >     /usr/lib/sendmail -or2h -q

- The queue can take a long time to process because too many messages are being queued unnecessarily. Several options affect the placement of mail messages into the queue. The `QueueLA` (`x`) option (see Section 34.8.50) tells sendmail to queue, rather than deliver, a message if the machine load is too high. Fewer messages will be queued if the value of that option is increased. The `SuperSafe` (`s`) option (see Section 34.8.67, SuperSafe (s)) tells sendmail to queue all messages for safety. If your machine "never" crashes, this may not be necessary. [6] The `HoldExpensive` (`c`) option (see Section 34.8.29, HoldExpensive (c)) tells sendmail to queue messages to "expensive" delivery agents (those with the `F=e` flag set; see Section 30.8.18, F=e) rather than delivering them. If the queue is routinely filled with messages to expensive sites, you should reconsider your reasons for marking those sites as expensive.

  > [6] Eric says, "Set this option. In fact, it may go away (or be ignored, i.e., always be on) in some future release."

- The queue can fill with messages because sendmail was run with the `-odq` or `-odd` command-line switch (see the `DeliveryMode` (`d`) option, Section 34.8.16). At sites that receive a great deal of UUCP mail for forwarding, the rmail(8) program is often set up to run sendmail in "queue only" mode with the `-odq` command-line switch. If UUCP mail is clogging your normal mail services, you should consider queueing it to a separate queue directory. You can then process that other directory with a separate queue run of sendmail. (Use of separate queue directories is discussed in Section 23.7, "Process Alternate Queues".)

- A slow machine can clog the queue. When a single machine is set up to handle the bulk of a site's mail, that machine should be as swift as possible. In general, a dedicated mail server should have a fast CPU with lots of memory. It should never allow users to log in to it, and it should run its own name server daemon.

### 23.6.2 From the Command Line

The `-q` command-line switch, invoked without a time interval argument, is used to run sendmail in queue-processing mode. In this mode, sendmail processes the queue once and then exits. This mode can be run interactively from the command-line or in the background via cron(8).

Other command-line switches can be combined with `-q` to refine the way the queue is processed. The `-v` (verbose) switch causes sendmail to print information about each message it is processing. The `-d` (debugging) switch may be used to produce additional information about the queue. We'll discuss the `-v` switch as it applies to the queue later in this chapter. Those `-d` debugging switches appropriate to the queue can be found in Table 37.2 of Section 37.5, "Reference in Numerical Order".

V8 sendmail allows variations on `-q`: `-qI` allows you to specify a specific message identifier for processing; `-qR` allows you to specify specific recipient addresses for processing; `-qS` allows you to specify specific sender addresses for processing. [7]

> [7] IDA and Pre-V8 SunOS sendmail offer three different command-line switches for processing the queue. The `-M` switch allows you to specify a specific message for processing. The `-R` switch allows you to specify specific recipient addresses for processing. The `-S` switch allows you to specify specific sender addresses for processing.

#### 23.6.2.1 Process the queue once: -q

The `-q` command-line switch, without an interval argument, tells sendmail to process the queue once, then exit. As such, this switch is a handy administrative tool. When the queue fills unexpectedly between queue runs of the daemon, for example, the `-q` command-line switch can be used to force an immediate queue run:

>     # /usr/lib/sendmail -q

On machines that do not run the sendmail daemon, the `-q` command-line switch can be used in conjunction with cron(8) to periodically process the queue. The following crontab(5) file entry, for example, causes sendmail to be run once per hour, at five minutes past the hour, to silently process the queue and exit:

>     5 * * * * /usr/lib/sendmail -q >/dev/null 2>&1

When used in conjunction with other switches (shown below), the `-q` switch allows many queue problems to be conveniently handled.

#### 23.6.2.2 Combine -v with -q

The `-q` switch without an argument prevents sendmail from running in the background and detaching from its controlling terminal. But it also runs silently. To see what is going on, use the `-v` command-line switch in combination with the `-q`:

>     % /usr/lib/sendmail -v -q

The `-v` command-line switch causes sendmail to print a step-by-step description of what it is doing. To illustrate, consider the following output produced by using both the `-v` and `-q` command-line switches:

>     Running MAA20989 (sequence 1 of 2)
>     <adams@dc.gov>... Connecting to dc.gov via ddn...
>     Trying 123.45.67.8... Connection timed out during user open with DC.GOV
>     <adams@dc.gov>... Deferred: Host DC.GOV is down

>     Running MAA27002 (sequence 2 of 2)
>     <help@irs.dc.gov>... Connecting to irs.dc.gov via ddn...
>     Trying 123.45.67.88...  connected.
>     220 irs.dc.gov Sendmail 5.57/3.0 ready at Mon, 27 Jan 92 09:16:38 -0400

Here, two queued messages are being processed. The first fails because of a connection timeout and is requeued for a later queue run. The second succeeds (we omit the full SMTP dialogue). After its delivery is complete, it is removed from the queue.

#### 23.6.2.3 Process by identifier/recipient/sender: -q[ISR]

With V8 sendmail you can process a subset of all queued messages. You can select which to process based on queue identifier, recipient address, or sender address:

>     -qIqueueid
>     -qRrecipient
>     -qSsender

The `-qI` variation is followed by a queue identifier such as KAA34556. The `-qR` is followed by the address of a recipient. The `-qS` is followed by the address of a sender. In all three variations there must be no space between the uppercase letter and the identifier or address.

These variations are used to limit the selection of queued files that are processed. For example:

>     % /usr/lib/sendmail -qSroot -qRbiff@here

Here, the queue is processed once. Only messages from root are processed. Of those, only messages that have biff@here as one of the recipients are processed.

In all three variations a partial specification of `queueid`, `recipient`, or `sender` is viewed by V8 sendmail as a substring. For example,

>     -qSroot

matches mail from all of the following:

>     root
>     ben@groots.edu
>     ben@GROOTS.EDU

The last line further illustrates that the substring match is a case-insensitive one. The substring match is literal. Wildcard characters (such as `*`) and regular expressions (such as `.*@.*edu`) won't work and may confuse the shell from which you run sendmail.

Multiple specifications may be combined on the command line (as shown above), but they all AND together:

>     % /usr/lib/sendmail -qI123 -qSroot -qR@host.edu

Here, the queue is processed only for messages with the number 123 anywhere in the message identifier that are also from root and that are also addressed to anyone at host.edu.

You can use the mailq command to preview the effect of these switches. For example, the following command will list (but not send) the messages that would be processed by the above command line:

>     % mailq -qI123 -qSroot -qR@host.edu

#### 23.6.2.4 Process the queue via ESMTP ETRN

The ESMTP ETRN command, based on RFC1985, causes V8.8 and above sendmail to asynchronously process its queue in a manner similar to the `-qR` command line switch (see Section 23.6.2.3, "Process by identifier/recipient/sender: -q[ISR]"). This new command allows dial-on-demand sites to make an SMTP connection and to force the other side to process and send any mail that is queued for them. The form of this ESMTP command looks like this:

>     ETRN hostname

If the host is missing, this error message will be returned:

>     550 Parameter required

Otherwise, the queue will be processed just as if the following command line argument were given:

>     -qR@hostname

In both cases a `qf` file will be processed if it has a host named `hostname` anywhere in the host part (following the `@`) of one of its `R` lines. The only difference here is that the former (the ETRN) operates asynchronously. That is, sendmail forks a copy of itself, and the forked child processes the queue.

One way to use `ETRN` is with a simple Bourne shell script:

>     #!/bin/sh
>     OURSITE=here.uofa.edu
>     MAILSERVER=mailhub.uofa.edu
>     TELNET=/usr/ucb/telnet
>     PORT=25

>     echo "etrn $OURSITE" | $TELNET $MAILSERVER $PORT
>     exit 0

If run on a dial-on-demand host, this script will force a dial-up network connection to be made. The mail server's sendmail is then forced, with ETRN, to process the queue and transport back any mail destined for the local host over the just-created network connection.

23.5 How the Queue Is Processed |  | 23.7 Process Alternate Queues